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SYSTEM AND METHOD FOR INSERTION OF RECORDED MEDIA 

INTO A BROADCAST 



FDZLD OF THE INVENTION 

The present invention relates generally to broadcast systems and, more 
particularly, to a system and method for inserting recorded media into a broadcast. 

BACKGROUND OF THE INVENTION 

There are numerous technologies available for inserting media into broadcast 
systems. For instance, public announcements generated as text may be added to a video 
broadcast signal (e.g., to notify viewers of technical difficulties, critical events, weather 
alerts, etc.). The scheduled program continues uninterrupted while an announcement text 
is displayed on the television screen. It is also possible to receive notification of a private 
event (such as receiving an e-mail) through a television screen. Notification may be 
generated as text and displayed on the television screen or may take the form of a 
recorded audio announcement. 

More particularly, in recent years the development of Germany's Verkehrsfunk 
(known broadly in Europe) has allowed people who listen to tapes or CD's to receive 
important announcements, generally relating to traffic information, without having to 
continually listen to a specific station just because it happens to periodically announce 
traffic information for the area. The system makes it possible, by means such as a simple 
tone decoder, to turn on and off the audio insert. However, Verkehrsfunk has some 
limitations. For one, the announcement interruption always takes precedence over the 
play of the recorded medium. The system is often so simplistic that the recording does 
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not even pause, but rather continues to play unheard while the announcement is spoken. 
The listener thus misses part of the recording. Moreover, the system uses a simple 
switch-over of the source selector of the amplifier, so the medium of the announcement 
and the medium of the recording (in this case, both audio) must be identical. 
5 In addition, the event driving the switch-over happens in real time. That is, the 

signal arrives and the event itself creates the signal. For example, when the speaker starts 
to make a traffic information announcement he pushes a button that turns on a first pilot 
tone and at the end of the announcement he pushes another button that creates an end 
pilot tone. 

0 In essence, to date the insertion of media into broadcast systems interrupts the 

broadcast stream. Moreover, prior systems are inefficient in that they require the user's 
medium of choice and the medium of the insertion to be of the same type. 
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SUMMARY OF THE INVENTION 

The present invention provides a system and method for prioritizing the insertion 
of recorded media into a broadcast stream according to a comparison of priority 
indicators in the broadcast stream and in the recorded media insertion. One embodiment 
5 provides a recorded media insertion that may be multi-media in nature. 

According to one embodiment, a broadcast stream is transmitted and received. 
At some point in time, an event occurs which requires the insertion of recorded media. If 
the recorded media insertion is a low priority the insertion may be delayed until the 
system determines that it will cause the least amount of disturbance. If the recorded 
w 10 media insertion is a high priority, it may be inserted immediately into the broadcast 
stream. In one embodiment, priority indicators may range between "0," the highest 
y, priority, and "n," the lowest priority. There may be a varying number of levels between 0 

■ and n, or only one level. 

D In one embodiment, a signal that is needed to change the priority of the event may 

H 15 be programmed by a time mark to synchronize with the broadcast stream. The signal 
y may be included in the broadcast stream as a pilot tone or by watermarking video, audio, 

or other media or data transmitted. The signal may also be transmitted over a separate 
network, again with a time mark to synchronize with the broadcast stream. In addition, if 
the watermark relates to video it may also contain an area in which an overlay may be 
20 played. Once the recorded media insertion has either finished by running its length or the 
priority level has been changed by sending another watermark, the system returns to its 
original broadcast-only type performance. 



In yet another alternative embodiment, the user's medium and the medium of the 
recorded insertion do not have to be of the same type (i.e., one could be video and the 
other could be audio, both could be video, or one could be video without audio and the 
other could be audio/video, etc.). 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a block diagram illustrating a prior art system for inserting an audio 
announcement over a radio system by interrupting the play of a cassette or CD. 

Figure 2 is a block diagram illustrating inserting recorded media into a broadcast 
stream in a television system in accordance with one embodiment of the present 
5 invention. 

Figure 3 is a flow chart describing the steps of comparing the priority indicators 
of a recorded media insertion with the priority indicators of a broadcast stream in 
accordance with one embodiment of the present invention. 

Figure 4 is a flow chart describing the steps of an Insertion Analyzer comparing 
10 the priority indicators of a broadcast stream the with priority indicators of multiple 
recorded media insertions in accordance with one embodiment. 

Figure 5 is a block diagram of a computer system in accordance with one 
embodiment of the present invention. 
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DETAILED DESCRIPTION 

Described herein is a system and method for inserting recorded media into a 
broadcast stream. Throughout the following description specific details are set forth in 
order to provide a more thorough understanding of the invention. However, the invention 
may be practiced without these particulars. In other instances, well known elements have 
5 not been shown or described in detail to avoid unnecessarily obscuring the present 
invention. Accordingly, the specification and drawings are to be regarded in an 
illustrative, rather than a restrictive, sense. 

One limitation of prior art techniques for media insertion into broadcast systems is 
that the insertion takes precedence over the play of the recorded medium. In cases where 
10 an announcement interrupts a recording, the listener misses part of the recording. It 
would be helpful if a system for media insertion into broadcast systems existed to 
prioritize the insertion according to priority indicators, such that the insertion may be 
delayed if it is a low priority and transmitted if it is a high priority. For example, if a user 
has been closely following a game show for nearly an hour and the answer to the million- 
15 dollar question finally arrives only to be obliterated by the voice-over "You've got mail," 
the user may be frustrated and upset. However, if a system existed to give the million- 
dollar answer a high priority level, the message "You've got mail" would have to wait 
until a low priority section of the broadcast stream. 

Another limitation of prior art techniques for media insertion into broadcast 
20 systems is that the medium of the announcement and the user's medium typically need to 
be identical (i.e., audio and audio, video and video, etc.). It would be helpful if a system 
existed for the insertion of multi-media into a broadcast system. 
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The system and method disclosed herein may be integrated into advanced 
Internet-or network-based knowledge systems as related to information retrieval, 
information extraction, and question and answer systems. Figure 5 is an example of one 
embodiment of a computer system 500. The system shown has a processor 501 coupled 

5 to a bus 502. Also shown coupled to the bus 502 are a memory 503 which may contain 
instructions 504 such as instructions for inserting media into a broadcast stream. 
Additional components shown coupled to the bus 502 are a storage device 505 (such as a 
hard drive, floppy drive, CD-ROM, DVD-ROM, etc.), an input device 506 (such as a 
keyboard, mouse, light pen, bar code reader, scanner, microphone, joystick, etc.), and an 

10 output device 507 (such as a printer, monitor, speakers, etc.). Of course, an exemplary 
computer system could have more components than these or a subset of the components 
listed. 

The system and method described herein may be stored in the memory of a 
computer system (i.e., a set-top box) as a set of instructions to be executed, as shown by 

15 way of example in Figure 5. In addition, the instructions to perform the system and 
method described herein may alternatively be stored on other forms of machine-readable 
media, including magnetic and optical disks. For example, the system and method of the 
present invention may be stored on machine-readable media, such as magnetic disks or 
optical disks, which are accessible via a disk drive (or computer-readable medium drive). 

20 Further, the instructions may be downloaded into a computing device over a data network 
in a form of a compiled and linked version. 

Alternatively, the logic to perform the system and method described herein may 
be implemented in additional computer and/or machine-readable media such as discrete 
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hardware components as large-scale integrated circuits (LSI's), application specific 
integrated circuits (ASIC's), firmware such as electrically erasable programmable read- 
only memory (EEPROM's), and electrical, optical, acoustical, and other forms of 
propogated signals (e.g., carrier waves, infrared signals, digital signals, etc.). 

5 Referring now to Figure 1 there is shown a block diagram illustrating inserting an 

audio announcement over a radio system 100 by interrupting the play of a cassette or CD. 
The stream of recording 101 may be a CD player, a cassette player, or any other device 
capable of reproducing audible sound from a recorded medium. The stream of recording 
101 is part of a radio system 100 including an AM/FM receiver (not shown in this view), 

10 a tuner for tuning to a radio station (not shown in this view), one or more speakers (not 
shown in this view) and various other components well known to those skilled in the art 
of audio systems. Announcement 111 may be received by an unused tuner in the radio 
system 100 that is playing the music and may contain signal tones such as tones 112 and 
116. When signal tones 112 and 116 turn on the announcement system, arrow 113 

15 indicates that recorded medium is suspended at point 114. The announcement between 
the tones 115 can then be heard by the listener, because it is inserted as indicated by 
arrow 120. At the end of the announcement, tone 116 then switches play back to the 
recorded medium, which event is marked at point 118, and the remainder of the recording 
continues to play on track 101. According to the illustration of Figure 1, the 

20 announcement interruption takes precedence over the play of the recorded medium. 
Moreover, the medium of the announcement and the medium of the recording (in this 
case both audio) need to be identical. 



Referring now to Figure 2 there is shown a block diagram illustrating inserting 
recorded media into a broadcast stream in a television system 200 in accordance with one 
embodiment. The present invention may be implemented in any television system 200 
including analog (e.g., using CRTs technology) as well as digital technologies (e.g., 
5 HDTV supporting interlaced format). It should be noted, however, that although 
television system 200 is used in the embodiment illustrated in Figure 2, any system 
capable of receiving broadcast stream 202 may be used, including radio systems. 
Television system 200 typically includes a receiver for receiving a broadcast signal, a 
display screen, a tuner for extracting a television signal corresponding to a channel, and 

10 other standard components well known to those skilled in the art of television systems. 

In the embodiment illustrated in Figure 2, television system 200 also includes a 
set-top box (not shown in this view). The set-top box is generally a user interface unit 
comprising a CPU coupled to a read-only memory (ROM) and a random-access memory 
(RAM)(not shown in this view). The ROM includes instructions and data for executing 

15 on the CPU. The RAM is used for storing program variables for the program instructions 
contained in the ROM. The set-top box of the present invention also contains executable 
code, typically in software, for determining at what point to insert recorded media 213 
into a broadcast stream 202 based on a comparison of priority indicators 240 in the 
recorded media insertion 213 and in the broadcast stream 202 in a manner that will be 

20 described in detail below. 

According to the embodiment illustrated in Figure 2, set-top box receives the 
broadcast stream 202 transmitted from a broadcast station 201 and delivers the broadcast 
stream 202 in real-time usable form to the receiver (not shown in this view) in the 



television system 200. At some point in time event 212 occurs, which requires the 
insertion of the recorded media 213 into the broadcast stream 202. The media may be a 
recorded announcement, overlay, text, video, or any other type of available media. The 
event 212 which triggers the recorded media insertion 213 into the broadcast stream 202 
5 may be a variety of events, such as notification to a user of the television system 200 that 
an e-mail message has arrived, information regarding traffic or weather conditions, the 
state of the stock market, sports updates, etc. The recorded media insertion 213 and the 
broadcast stream 202 are assigned priority indicators 240 ranging between "0," the 
highest priority, and "n " the lowest priority. There may be any number of levels 

10 between 0 and n, or only one level. Of course, priority indicators 240 do not necessarily 
have to range between 0 and n. In another embodiment, for instance, priority indicators 
240 may range from "1" through "9," "A" through "Z," or may even be symbols such as 
for the highest priority and "-" for the lowest priority. Priority indicators 240 may be 
embedded into the broadcast stream 202 according to numerous known technologies, 

15 including a pilot tone or by watermarking either video, audio, or other media or data 
transmitted. A signal (not shown in this view) that is needed to change the priority 
indicators 240 may be programmed by a time mark. In this way, the recorded media 
insertion 213 synchronizes with the broadcast stream 202. It should also be noted that it 
is possible to use existing transport streams (such as the transport streams in digital 

20 television broadcasts) to embed the priority indicators 240 into the broadcast stream 202. 

In a further aspect, the priority indicators 240 of various components of the 
broadcast stream 202 may be annotated by a broadcaster. For instance, the broadcaster 
may assign different priority indicators 240 to various segments of a newscast, such as 



priority indicator 0 to the actual program and priority indicator n to the commercials. 
Advertisers could also use the system for low-level local advertising during the newscast 
(or other program), for example, by allowing an advertiser to address just one subdivision 
of a population group or some geographic area located close to a specific store or service 
5 place. This kind of micro-advertising would allow broadcasters to sell inexpensive local 
advertising to businesses, which otherwise would not have a chance to get onto a major 
channel during prime-time. It is also possible for the user to assign priority indicators 240 
to the recorded media insertion 213. For instance, the user may determine that all e-mail 
notifications from family members receive priority indicator 0 and all e-mail notifications 

10 from friends receive priority indicator n. The user may set for each level and each media 
type an ignore or a higher level to replace the broadcast stream, thus allowing the user to 
customize the experience. 

In a further aspect of the present invention, priority indicators 240 in the broadcast 
stream 202 may be transmitted over a separate network (e.g., the Internet) (not shown in 

15 this view). In this embodiment, set-top box is connected to a server (not shown in this 
view) through the separate network. The server receives the priority indictor 240 of the 
broadcast stream 202 through the separate network and transmits the priority indicator 
240 to the set-top box, rather than having the priority indicators 240 be embedded into the 
broadcast stream 202 through a watermark or some other method. In addition, the 

20 recorded media insertion 213 may either be transmitted in advance, using the same 
transport as is used by the program information, or may be transmitted over the separate 
network. 
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When watermark 220 arrives, vector 221 changes the priority indicator 240 at 
point 241 to level 1, which in turn releases a prerecorded announcement 242 for play by 
sending command 222 and suspending the replay or reducing the replay of the broadcast 
stream 202. Prerecorded announcement 242 is then inserted or overlaid, depending again 
5 on the medium type and the announcement content. Prerecorded announcement 242 may 
be, for example, a voice-over such as "You've got mail," that plays during a speaking 
pause in the program or it may be an overlay in the background of a video such as "Order 
your movie tickets now ~ Click here," generated as text and displayed on the television 
screen. Once the recorded media insertion 213 has either finished by running its length 

10 or the priority indicator 240 has been changed by sending a second watermark 225, the 
television system 200 returns to its original broadcast-only type performance as indicated 
by arrows 226, 227, and 242 a through n. The earlier event 242a indicates that the media 
insertion 213 has stopped before the second watermark 225 is sent. 

Non-conventional media may also be used. For example, odor media, not 

15 currently available, may be used to allow broadcasters to combine smells in broadcasts. 
Odor media may be used to advertise products (such as perfume, for instance, or pizza), 
to notify a user that their restaurant take-out meal is available, or in any other capacity 
where the release of an odor may be helpful. Also, the recorded media insertion 213 may 
be multi-media in nature. Depending on the required interrupt priority indicator 240, 

20 some recorded media insertion 213 may be played, while others may be held back. For 
example, "You've got mail" may be both a video clip and a sound file. Depending on the 
current priority indicator 240 of the broadcast stream 202 and on the priority indicator 
240 assigned to the recorded media insertion 213, for example, only the sound or only the 



video would play, rather than both or none. Also, if there is no break in the priority of 
interrupt requests, a queue of recorded media insertions 213 may build, which eventually 
may be played, one at a time, or in a multiple view screen, showing all the icons/media at 
the same time, based on user requests. 
5 In addition, all kinds of variations may be used for the user configurations and 

broadcast inserts as well as for the recordings, and all kinds of compression techniques, 
bitmapping techniques, 3-D texture mapping, etc. 

Referring now to Figure 3, there is shown a flow chart illustrating executing code 
to compare the priority indicator of a recorded media insertion with the priority indicator 

10 in a broadcast stream 300 in accordance with one embodiment. Broadcast stream 310 
(e.g., a TV program) in any time frame has an assigned priority indicator A and recorded 
media insertion 320 has an assigned priority indicator B. Both priority indicator A and 
priority indicator B are functions of time A(t) and B(t). The executing code 330 
compares A(t) and B(t) (processing block 340). If A(t) is greater than B(t) then the 

15 broadcast stream is not interrupted. Otherwise, an additional recorded media insertion 
350 is superimposed on the broadcast stream. 

Referring now to Figure 4, there is shown a flow chart of the steps for comparing 
priority indicators of a broadcast stream with priority indicators of multiple recorded 
media insertions 400 in accordance with one embodiment. One or more events 401 that 

20 require media insertions into a broadcast stream 402 trigger the initiation of the routine 
400 (for example, through an interrupt as is well known in the art). At processing block 
403 priority indicators for events 401 that trigger media insertions are called up. For 
instance, e-mail notifications may have a low priority indicator and phone call 



notifications may have a high priority indicator. Priority indicators of the media 
insertions are decoded as are the priority indicators of the broadcast stream 402 
(processing block 404). 

The routine 400 applies filter criteria (processing block 405) where a user may set 
5 their own preferences for events 401 triggering the media insertions (processing block 
405). For instance, the user may give the highest priority indicator to media insertions 
for an event caused by a particular person causing the event (for instance, the highest 
priority indicator may be given to all e-mail notifications and phone call notifications 
from a user's spouse). At processing block 406 the routine 400 sorts media insertions by 

10 priority indicators, such that media insertions go through a loop (processing blocks 407 
and 408). The loop compares the priority indicators of multiple media insertions against 
the priority indicators of the broadcast stream 402. The loop allows for media insertions 
into the broadcast stream 402 until an event with a media insertion with a lower priority 
than the broadcast stream 402 appears. The system 400 sorts (processing block 406) the 

15 media insertions by placing media insertions with the highest priority first. When all the 
media insertions have played, the routine interrupts (processing block 430) and a CPU 
(not shown in this view) that contains routine 400 returns to its original function. 

Thus, a system and method for inserting recorded media into a broadcast stream 
has been described. Although the foregoing description and accompanying figures 

20 discuss and illustrate specific embodiments, it should be appreciated that the present 
invention is to be measured only in terms of the claims that follow. 
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CLAIMS 



What is claimed is: 

1 A system, comprising 

2 a receiver to receive a broadcast stream and recorded media; and 

3 a unit to compare a priority indicator in the broadcast stream and a priority 

4 indicator in the recorded media, the unit permitting insertion of the recorded media into 

5 the broadcast stream when the priority indicators in the recorded media are higher than in 

6 the broadcast stream and delaying insertion when the priority indicators in the recorded 

7 media are lower than in the broadcast stream. 

1 2. The system of Claim 1 wherein the priority indicators are numbers, letters, or 

2 symbols in the broadcast stream and in the recorded media. 

1 3. The system of Claim 1 wherein the recorded media and the broadcast stream are 

2 the same media. 

1 4. The system of Claim 1 wherein the recorded media and the broadcast stream are 

2 different media. 

1 5. The system of Claim 1 wherein an event triggers the recorded media insertion into 

2 the broadcast stream. 
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1 6. The system of Claim 5 wherein the event that triggers the recorded media 

2 insertion into the broadcast stream includes notification that an e-mail message has 

3 arrived. 

1 7. The system of Claim 1 wherein a signal that is needed to change the priority 

2 indicator of the media insertion is programmed by a time mark, the time mark 

3 synchronizing the recorded media insertion with the broadcast stream. 

1 8. The system of Claim 1 wherein the priority indicators are embedded into the 

2 broadcast stream using a pilot tone or watermark. 

1 9. The system of Claim 1 wherein the priority indicators are assigned to the recorded 

2 media insertions by a user of the system. 

1 10. The system of Claim 1 wherein the receiver is part of a television system or radio 

2 system. 

1 y/ A method of inserting recorded media into a broadcast stream, the method 

2 comprising: 

3 transmitting a broadcast stream and recorded media to a receiver of a broadcast 

4 system, the broadcast stream and the recorded media containing one or more priority 

5 indicators; 

6 comparing priority indicators in the broadcast stream with priority indicators in 

7 the recorded media; and 



8 inserting the recorded media into the broadcast stream when the priority 

9 indicators in the recorded media are higher than in the broadcast stream and delaying 

10 inserting the recorded media into the broadcast stream when the priority indicators in the 

1 1 recorded media are lower than in the broadcast stream. 

1 12. The method of Claim 11 wherein the priority indicators are numbers, letters, or 

2 symbols in the broadcast stream and in the recorded media. 

1 13. The method of Claim 1 1 wherein the recorded media and the broadcast stream are 

2 the same media. 

1 14. The method of Claim 1 1 wherein the recorded media and the broadcast stream are 

2 different media. 

1 15. The method of Claim 1 1 wherein an event triggers the recorded media insertion 

2 into the broadcast stream. 

1 16. The method of Claim 15 wherein the event that triggers the recorded media 

2 insertion into the broadcast stream includes notification that an e-mail message has 

3 arrived. 

1 17. The method of Claim 1 1 wherein a broadcaster may assign a plurality of priority 

2 indicators to different segments of a broadcast stream, the priority indicators embedded 

3 into the broadcast stream using a pilot tone or watermark. 



18. The method of Claim 1 1 wherein a broadcaster may assign a plurality of priority 
indicators to different segments of a broadcast stream based on a subdivision or 
geographic area, the plurality of priority indicators allowing broadcasters to sell 
advertising space for the particular subdivision or geographic area. 

19. The method of Claim 1 1 wherein a signal used to change the priority indicator of 
the media insertion is programmed by a time mark, the time mark synchronizing the 
recorded media insertion with the broadcast stream. 



a server; 

a set-top box coupled to the server; 

a receiver in the set-top box to receive a broadcast stream and recorded media; 

a transmitter in the receiver to transmit the broadcast stream and the recorded 
media to a television receiver; and 

a unit in the set-top box to compare one or more priority indicators in the 
broadcast stream and in the recorded media, the unit permitting insertion of the recorded 
media into the broadcast stream when the priority indicators in the recorded media are 
higher than in the broadcast stream and delaying insertion when the priority indicators in 
the recorded media are lower than in the broadcast stream. 

21 . The system of Claim 20 wherein the priority indicators are numbers, letters, or 
symbols in the broadcast stream and in the recorded media. 




A system, comprising: 
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22. The system of Claim 20 wherein the recorded media and the broadcast stream are 
the same media. 



1 23. The system of Claim 20 wherein the recorded media and the broadcast stream are 

2 different media. 

1 24. The system of Claim 20 wherein an event triggers the recorded media insertion 

2 into the broadcast stream. 

1 25. The system of Claim 24 wherein the event that triggers the recorded media 

2 insertion into the broadcast stream includes notification that an e-mail message has 

3 arrived. 

1 h machine-readable storage medium tangibly embodying a sequence of 

2 instructions executable by the machine to perform a method for inserting recorded media 

3 into a broadcast stream, the method comprising: 

4 transmitting a broadcast stream and recorded media to a receiver of a broadcast 

5 system, the broadcast stream and the recorded media containing one or more priority 

6 indicators; 

7 comparing priority indicators in the broadcast stream with priority indicators in 

8 the broadcast media; and 

9 inserting the recorded media into the broadcast stream when the priority 

10 indicators in the recorded media are higher than in the broadcast stream and delaying 

1 1 inserting the recorded media into the broadcast stream when the priority indicators in the 

12 recorded media are lower than in the broadcast stream. 
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1 27. The machine-readable storage medium of Claim 26 wherein the priority indicators 

2 are numbers, letters, or symbols in the broadcast stream and in the recorded media. 

1 28. The machine-readable storage medium of Claim 26 wherein the recorded media 

2 and the broadcast stream are the same media. 

1 29. The machine-readable storage medium of Claim 26 wherein the recorded media 

2 and the broadcast stream are different media. 

1 30. The machine-readable storage medium of Claim 26 wherein an event triggers the 

2 recorded media insertion into the broadcast stream. 

1 31. The machine-readable storage medium of Claim 30 wherein the event that 

2 triggers the recorded media insertion into the broadcast stream includes notification that 

3 an e-mail message has arrived. 

1 32. The machine-readable storage medium of Claim 26 wherein a broadcaster may 

2 assign a plurality of priority indicators to different segments of a broadcast stream, the 

3 priority indicators embedded into the broadcast stream using a pilot tone or watermark. 

1 33. The machine-readable storage medium of Claim 26 wherein a broadcaster may 

2 assign a plurality of priority indicators to different segments of a broadcast stream based 

3 on a subdivision or geographic area, the plurality of priority indicators allowing 

4 broadcasters to sell advertising space for the particular subdivision or geographic area. 



1 



2 



34. The machine-readable storage medium of Claim 26 wherein the priority indicators 
are assigned to the recorded media insertions by a user of the system. 



1 35. The machine-readable storage medium of Claim 26 wherein a signal used to 

2 change the priority indicator of the media insertion is programmed by a time mark, the 

3 time mark synchronizing the recorded media insertion with the broadcast stream. 
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ABSTRACT 



A system and method for prioritizing the insertion of recorded media into a broadcast 
stream according to a comparison of priority indicators in the broadcast stream and in the 
recorded media insertion. The recorded media insertion may be multi-media in nature. 



22 




ft 




8 



Attorney's Docket No.: 004688. P021 Patent 
DECLARATION AND POWER OF ATTORNEY FOR PATENT APPLICATION 



As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below, next to my name. 

I believe I am the original, first, and sole inventor (if only one name is listed below) or an original, 
first, and joint inventor (if plural names are listed below) of the subject matter which is claimed and 
for which a patent is sought on the invention entitled 

Dan Kikinis and Yakov Kamen 



the specification of which 

X is attached hereto. 

was filed on as 

United States Application Number 

or PCT International Application Number 

and was amended on . 

(if applicable) 

I hereby state that I have reviewed and understand the contents of the above-identified 
specification, including the claim(s), as amended by any amendment referred to above. I do not 
know and do not believe that the claimed invention was ever known or used in the United States of 
America before my invention thereof, or patented or described in any printed publication in any 
country before my invention thereof or more than one year prior to this application, that the same 
was not in public use or on sale in the United States of America more than one year prior to this 
application, and that the invention has not been patented or made the subject of an inventor's 
certificate issued before the date of this application in any country foreign to the United States of 
America on an application filed by me or my legal representatives or assigns more than twelve 
months (for a utility patent application) or six months (for a design patent application) prior to this 
application. 

I acknowledge the duty to disclose all information known to me to be material to patentability as 
defined in Title 37, Code of Federal Regulations, Section 1 .56. 

I hereby claim foreign priority benefits under Title 35, United States Code, Section 1 19(a)-(d), of any 
foreign application (s) for patent or inventor's certificate listed below and have also identified below 
any foreign application for patent or inventor's certificate having a filing date before that of the 
application on which priority is claimed: 
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Prior Foreign Application(s) 



Priority 
Claimed 



(Number) (Country) (Day/Month/Year Filed) Yes No 



(Number) (Country) (Day/Month/Year Filed) Yes No 



(Number) (Country) (Day/Month/Year Filed) Yes No 

I hereby claim the benefit under title 35, United States Code, Section 1 19(e) of any United States 
provisional application(s) listed below: 



(Application Number) Filing Date 



(Application Number) Filing Date 



I hereby claim the benefit under Title 35, United States Code, Section 120 of any United States 
application(s) listed below and, insofar as the subject matter of each of the claims of this application 
is not disclosed in the prior United States application in the manner provided by the first paragraph 
of Title 35, United States Code, Section 1 12, 1 acknowledge the duty to disclose all information 
known to me to be material to patentability as defined in Title 37, Code of Federal Regulations, 
Section 1 .56 which became available between the filing date of the prior application and the national 
or PCT international filing date of this application: 



(Application Number) Filing Date (Status -- patented, 

pending, abandoned) 



(Application Number) Filing Date (Status patented, 

pending, abandoned) 

I hereby appoint the persons listed on Appendix A hereto (which is incorporated by reference and a 
part of this document) as my respective patent attorneys and patent agents, with full power of 
substitution and revocation, to prosecute this application and to transact all business in the Patent 
and Trademark Office connected herewith. 

Send correspondence to John P. Ward , BLAKELY, SOKOLOFF, TAYLOR & 

(Name of Attorney or Agent) 
ZAFMAN LLP, 12400 Wilshire Boulevard 7th Floor, Los Angeles, California 90025 and direct 

telephone calls to John P. Ward , (408) 720-8300. 

(Name of Attorney or Agent) 
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I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these 
statements were made with the knowledge that willful false statements and the like so made 
are punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United 
States Code and that such willful false statements may jeopardize the validity of the 
application or any patent issued thereon. 

Full Name of Sole/First Inventor Dan Kikinis 

Inventor's Signature . Date 

Residence Saratoga. California Citizenship _ 

(City, State) (Country) 

Post Office Address - — 



Full Name of Second/Joint Inventor Yakov Kamen . 

Inventor's Signature _ Date 

Residence Cupertino, California Citizenship 

(City, State) (Country) 

Post Office Address „ 



Full Name of Third/Joint Inventor _ _ 

Inventor's Signature _ Date 

Residence . Citizenship _ — 

(City, State) (Country) 

Post Office Address „ „ — 



Full Name of Fourth/Joint Inventor „ . 

Inventor's Signature Date 

Residence . Citizenship _ 

(City, State) (Country) 

Post Office Address „ » — 
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APPENDIX A 



William E. Alford, Reg. No. 37,764; Farzad E. Amini, Reg. No. P42,261 ; Aloysius T. C. AuYeung, Reg. No. 
35,432; William Thomas Babbitt, Reg. No. 39,591; Carol F. Barry, Reg. No. 41,600; Jordan Michael 
Becker, Reg. No. 39,602; Lisa N. Benado, Reg. No. 39,995; Bradley J. Bereznak, Reg. No. 33,474; 
Michael A. Bernadicou, Reg. No. 35,934; Roger W. Blakely, Jr., Reg. No. 25,831; R. Alan Burnett, Reg. 
No. 46,149; Gregory D. Caldwell, Reg. No. 39,926; Andrew C. Chen, Reg. No. 43,544; Thomas M. 
Coester, Reg. No. 39,637; Donna Jo Coningsby, Reg. No. 41,684; Florin Corie, Reg. No. 46,244; Dennis 
M. deGuzman, Reg. No. 41,702; Stephen M. De Klerk, Reg. No. P46,503; Michael Anthony DeSanctis, 
Reg. No. 39,957; Daniel M. De Vos, Reg. No. 37,813; Robert Andrew Diehl, Reg. No. 40,992; Sanjeet 
Dutta, Reg. No. P46,145; Matthew C. Fagan, Reg. No. 37,542; Tarek N. Fahmi, Reg. No. 41,402; George 
Fountain, Reg. No. 37,374; Paramita Ghosh, Reg. No. 42,806; James Y. Go, Reg. No. 40,621; James A. 
Henry, Reg. No. 41,064; Libby N. Ho, Reg. No. P46,774; Wilimore F. Hoibrow III, Reg. No. P41 ,845; 
Sheryl Sue Holloway, Reg. No. 37,850; George W Hoover II, Reg. No. 32,992; Eric S. Hyman, Reg. No. 
30,139; William W. Kidd, Reg. No. 31,772; Sang Hui Kim, Reg. No. 40,450; Walter T. Kim, Reg. No. 
42,731; Eric T. King, Reg. No. 44,188; Erica W. Kuo, Reg. No. 42,775; George Brian Leavell, Reg. No. 
45,436; Kurt P. Leyendecker, Reg. No. 42,799; Gordon R. Lindeen III, Reg. No. 33,192; Jan Carol Little, 
Reg. No. 41,181; Joseph Lutz, Reg. No. 43,765; Michael J. Mallie, Reg. No. 36,591; Andre L. Marais, 
under 37 C.F.R. § 10.9(b); Paul A. Mendonsa, Reg. No. 42,879; Clive D. Menezes, Reg. No. 45,493; 
Chun M. Ng, Reg. No. 36,878; Thien T. Nguyen, Reg. No. 43,835; Thinh V. Nguyen, Reg. No. 42,034; 
Dennis A. Nicholls, Reg. No. 42,036; Daniel E. Ovanezian, Reg. No. 41,236; Kenneth B. Paley, Reg. No. 
38,989; Marina Portnova, Reg. No. P45,750; William F. Ryann, Reg. 44,313; James H. Salter, Reg. No. 
35,668; William W. Schaal, Reg. No. 39,018; James C. Scheller, Reg. No. 31,195; Jeffrey Sam Smith, 
Reg. No. 39,377; Maria McCormack Sobrino, Reg. No. 31,639; Stanley W. Sokoloff, Reg. No. 25,128; 
Judith A. Szepesi, Reg. No. 39,393; Vincent P. Tassinari, Reg. No. 42,179; Edwin H. Taylor, Reg. No. 
25,129; John F. Travis, Reg. No. 43,203; Joseph A. Twarowski, Reg. No. 42,191; Tom Van Zandt, Reg. 
No. 43,219; Lester J. Vincent, Reg. No. 31 ,460; Glenn E. Von Tersch, Reg. No. 41 ,364; John 
Patrick Ward, Reg. No. 40,216; Mark L Watson, Reg. No. P46,322; Thomas C. Webster, Reg. No. 
P46,154; Steven D. Yates, Reg. No. 42,242; and Norman Zafman, Reg. No. 26,250; my patent attorneys, 
and Firasat Ali, Reg. No. 45,715; and Justin M. Dillon, Reg. No. 42,486; my patent agents, of BLAKELY, 
SOKOLOFF, TAYLOR & ZAFMAN LLP, with offices located at 12400 Wilshire Boulevard, 7th Floor, 
Los Angeles, California 90025, telephone (310) 207-3800, and James R. Thein, Reg. No. 31 ,710, my 
patent attorney with full power of substitution and revocation, to prosecute this application and to transact 
all business in the Patent and Trademark Office connected herewith. 
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APPENDIX B 



Title 37, Code of Federal Regulations, Section 1 .56 
Duty to Disclose Information Material to Patentability 

(a) A patent by its very nature is affected with a public interest. The public interest is best served, 
and the most effective patent examination occurs when, at the time an application is being examined, the 
Office is aware of and evaluates the teachings of all information material to patentability. Each individual 
associated with the filing and prosecution of a patent application has a duty of candor and good faith in 
dealing with the Office, which includes a duty to disclose to the Office all information known to that individual 
to be material to patentability as defined in this section. The duty to disclosure information exists with respect 
to each pending claim until the claim is cancelled or withdrawn from consideration, or the application becomes 
abandoned. Information material to the patentability of a claim that is cancelled or withdrawn from 
consideration need not be submitted if the information is not materia! to the patentability of any claim 
remaining under consideration in the application. There is no duty to submit information which is not material 
to the patentability of any existing claim. The duty to disclosure all information known to be material to 
patentability is deemed to be satisfied if all information known to be material to patentability of any claim 
issued in a patent was cited by the Office or submitted to the Office in the manner prescribed by §§1 .97(b)-(d) 
and 1 .98. However, no patent will be granted on an application in connection with which fraud on the Office 
was practiced or attempted or the duty of disclosure was violated through bad faith or intentional misconduct. 
The Office encourages applicants to carefully examine: 

(1 ) Prior art cited in search reports of a foreign patent office in a counterpart application, and 

(2) The closest information over which individuals associated with the filing or prosecution of a 
patent application believe any pending claim patentably defines, to make sure that any material information 
contained therein is disclosed to the Office. 

(b) Under this section, information is material to patentability when it is not cumulative to 
information already of record or being made or record in the application, and 

(1) It establishes, by itself or in combination with other information, a prima facie case of 
unpatentability of a claim; or 

(2) It refutes, or is inconsistent with, a position the applicant takes in: 

(i) Opposing an argument of unpatentability relied on by the Office, or 

(ii) Asserting an argument of patentability. 

A prima facie case of unpatentability is established when the information compels a conclusion that a claim is 
unpatentable under the preponderance of evidence, burden-of-proof standard, giving each term in the claim 
its broadest reasonable construction consistent with the specification, and before any consideration is given to 
evidence which may be submitted in an attempt to establish a contrary conclusion of patentability. 

(c) Individuals associated with the filing or prosecution of a patent application within the 
meaning of this section are: 

(1) Each inventor named in the application; 

(2) Each attorney or agent who prepares or prosecutes the application; and 

(3) Every other person who is substantively involved in the preparation or prosecution of the 
application and who is associated with the inventor, with the assignee or with anyone to whom there is an 
obligation to assign the application. 

(d) Individuals other than the attorney, agent or inventor may comply with this section by 
disclosing information to the attorney, agent, or inventor. 
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